Exit Time Analysis for Approximations of Gradient Descent Trajectories Around Saddle Points

Abstract This paper considers the problem of understanding exit time for trajectories gradient-related first-order methods from saddle neighborhoods under some initial boundary conditions. Given ‘flat’ geometry around points, can struggle to escape these regions in a fast manner due small magnitudes gradients encountered. In particular, while it is known that strict-saddle neighborhoods, existing analytic techniques do not explicitly leverage local points order control behavior gradient trajectories. It this context puts forth rigorous geometric analysis gradient-descent method using matrix perturbation theory. doing so, provides key result be used generate an approximate trajectory any given addition, leads linear exit-time solution certain necessary conditions, which bring out dependence on dimension, conditioning neighborhood, and more, class functions.